1. Field of the Invention
The present invention relates to an image processing apparatus, and particularly to an image processing apparatus which can easily generate a variety of signals as key signals used at mixing of images.
2. Description of the Related Art
FIG. 1 shows a structure of an example of a conventional video switcher as an image mixing system which mixes two images in accordance with a key signal and outputs a mixed image.
In this video switcher, a key signal which controls a ratio of mixing (transmittance of one image to the other image when the two images are mixed) for every pixel and has a value within the range of from 0 to 1, is generated, and images A and image B are mixed in accordance with the key signal.
That is, an input unit 1 includes a button, a dial, a lever and the like for specifying patterns and parameters of various wipes. When a user operates the button and the like for a pattern and parameter of a desired wipe, a signal corresponding to the operation is outputted to a controller 31. When receiving the signal from the input unit 1, the controller 31 controls a key signal generating unit 32 based on the signal, so that the key signal generating unit 32 is made to generate a key signal for executing the wipe. This key signal is supplied to a mixing unit (image mixing unit) 4. The image mixing unit 4 is supplied with not only the key signal but also two images A and B as objects of mixing. When receiving the key signal from the key signal generating unit 32, the image mixing unit 4 mixes the images A and B in accordance with the key signal and outputs a mixed image C obtained as the result of mixing.
FIG. 2 shows a structural example of the key signal generating unit 32 in the video switcher of FIG. 1.
In FIG. 2, the key signal generating unit 32 is constituted by a command buffer 41, a general-purpose CPU (Central Processing Unit) 42, a program memory 43, and a key signal storing memory 44.
The command buffer 41 is supplied, from the controller 31, with a command to the general-purpose CPU 42 and other data necessary for generating a key signal, and the command buffer 41 temporarily stores the command or the like from the controller 31. The general-purpose CPU 42 reads the command stored in the command buffer 41, and executes one of the programs stored in the program memory (internal ROM (Read Only Memory) 43, which corresponds to the command read from the command buffer 41. By this, the general-purpose CPU 42 generates a key signal, and stores the key signal corresponding to each pixel in the key signal storing memory 44. The key signal storing memory 44 is composed of, for example, a dual-port memory with storage capacity corresponding to pixels of one screen, temporarily stores key signals for the respective pixels of one screen from the general-purpose CPU 42, and supplies them to the mixing unit 4. Incidentally, the program memory 43 can be made built in the general-purpose CPU 42.
Next, FIG. 3 shows a structural example of the controller 31 in the video switcher of FIG. 1.
The controller 31 is mainly constituted by a CPU 51 and a program memory 52. The CPU 51 executes a program stored in the program memory 52 in response to a signal from the input unit 1, so that it performs a process for controlling the key signal generating unit 32.
That is, the program memory 52 stores a key selecting program for carrying out a key selecting process. The CPU 51 executes the key selecting program in accordance with a signal from the input unit 1, so that it performs the key selecting process for generating a key number (key No.) which identifies a shape pattern of a key signal to be generated by the key signal generating unit 32, and for generating a parameter which specifies the state of the key signal (for example, size, position, etc. of the key signal) and the way of changing the key signal (rotation angle, amount of parallel movement, etc.). The CPU 51 supplies, as a command, the key number and parameter obtained as the result of the key selecting process to the key signal generating unit 32.
The program memory 43 of the key signal generating unit 32 stores key signal generating programs for performing key signal generating processes. That is, the program memory 43 stores the key signal generating programs each generating a key signal of a shape pattern corresponding to each key number. When receiving a key number and parameter from the controller 31, the CPU 42 of the key signal generating unit 32 executes the key signal generating program corresponding to the key number in accordance with the parameter from the controller 31. By this, a key signal in which its shape pattern is specified by the key number and its state and way of changing are specified by the parameter, is generated and is stored in the key signal storing memory 44.
Next, with reference to the flowchart of FIG. 4, the key selecting process executed by the CPU 51 of FIG. 3 will be described further.
In the key selecting process, when the user operates the input unit 1 and a signal corresponding to the operation is supplied to the CPU 51, at step S31, the CPU 51 determines the shape pattern of the key signal in accordance with the signal, and further, at step S32, determines the parameter for specifying the state of the key signal and the way of changing the key signal. Then, the CPU 51 proceeds to step S33, and transmits the key number, which corresponds to the shape pattern of the key signal determined at step S31, and the parameter determined at step S32 to the key signal generating unit 32. Next, the CPU waits for supply of a signal corresponding to operation of the user from the input unit 1 and returns to step S31.
Next, with reference to the flowchart of FIG. 5, the key signal generating process performed by the CPU 42 of FIG. 3 (FIG. 2) will be further described.
In the key signal generating process, first of all, at step 41, the CPU 42 receives the key number and the parameter transmitted from the controller 31, proceeds to step S42, and reads out one (hereinafter, referred to suitably as individual pattern generating program) of the key signal generating programs stored in the program memory 43 which corresponds to the key number from the controller 31. Further, at step S42, the CPU 42 executes the individual pattern generating program in accordance with the parameter from the controller 31. By this, the CPU 42 generates such a key signal that its shape pattern is specified by the key number and its state and way of changing are specified by the parameter. The CPU 42 stores this key signal in the key signal storing memory 44, and then, waits for a key number and parameter transmitted from the controller 31 and returns to step S41.
Next, with reference to FIG. 6, the process of the mixing unit 4 of FIG. 1 will be described.
For example, in the key signal generating unit 32, when a star-shaped key signal K is generated, and is inputted, together with images A and B, into the mixing unit 4, the images A and B are mixed in the mixing unit 4 in accordance with the key signal K so that a mixed image C is formed.
Now, it is assumed that a pixel at an x-th position from the left of a screen and at a y-th position from the above is expressed by p(x, y), and pixel values of the images A, B, and C at the pixel p(x, y) are expressed by a(x, y), b (x, y), and c(x, y), respectively. Besides, it is assumed that a value of the key signal K at the pixel p(x, y) is expressed by k(x, y). When the key signal k(x, y) is expressed by, for example, 8 bits, in the mixing unit 4, the pixel value c(x, y) of the image C is obtained in accordance with, for example, the following equation.
c(x, y)=(k(x, y)a(x, y)+(255xe2x88x92k(x, y))b(x, y))/255xe2x80x83xe2x80x83(1)
In the key signal K, the value k(x, y) of a white star-shaped portion corresponds to 255, and the value k(x, y) of a black portion corresponds to 0, respectively. By this, in the mixed image C, the image A appears at the portion corresponding to the white star-shaped portion in the key signal K, and the image B appears at the portion corresponding to the black portion in the key signal K.
In the case shown in FIG. 6, for example, if the area of the white star-shaped portion in the key signal K is made gradually small, in the mixed image C, the portion where the image A is displayed becomes gradually small, and finally, only the image B is displayed. Thus, by changing the key signal K in such a manner, it is possible to realize a wipe process in which display portions of the images A and B are divided by the star, and the image A is gradually changed to the image B. Incidentally, in FIG. 6, although the shape of the key signal K is made star-shaped, even if other shapes are used, a similar wipe process can be carried out.
Besides, for example, as shown in FIG. 7A, in the case where the right side or left side of the key signal K is made 0 or 255, and the boundary portion between 0 and 255 is gradually moved from the right to left, as shown in FIG. 7B, it is possible to perform such a wipe process that the image A is gradually changed to the image B from the right to the left. Incidentally, if the boundary portion between 0 and 255 in the key signal K is, for example, gradually changed from 0 to 255, or is made 128 or the like of the intermediate value, the boundary portion between the image A and the image B becomes such that the two images A and B become semitransparent and are mixed to each other.
Further, for example, as shown in FIG. 8, if a key signal K of such an animation that a star rises from the depth side to this side is generated, for example, it is possible to perform such a wipe process that the image B divided by the star rises from the image A.
Besides, for example, if the values of the key signals K for all pixels are gradually changed from 255 to 0, it is possible to realize a so-called gradation wipe such that only the image A is first displayed, then the images A and B become semitransparent to form a blended state, and only the image B is finally displayed.
As described above, a wipe can be realized by mixing the images A and B while changing the values of the key signals K for the respective pixels. Thus, it is necessary to change the key signal K in real time in order to carry out such a wipe process that an image is blended with another image and finally changed to the other image, or another image appears from the center portion of one image, its area is gradually increased, and finally, the one image is changed to the other image.
However, for generating the key signal, it takes a time corresponding to the complexity and the like of a wipe which is to be realized by the key signal. Thus, patterns of wipes which the video switcher can realize with the key signals which can be generated by the CPU 42 of the key signal generating unit 32 in real time, are restricted. Specifically, in the case where a memory with capacity for one screen is used for the key signal storing memory 44, the key signal must be generated in the vertical return line interval. Even in the case where a memory with capacity for two screens is used, as a so-called double buffer structure, for the key signal storing memory 44, the key signal must be generated in a time of one screen (in a time from a vertical synchronizing signal to a next vertical synchronizing signal).
If a CPU with higher speed is used as the CPU 42, it is possible to attain high speed generation of the key signal. However, the CPU 42 is generally a general-purpose CPU, and does not have a specific function for writing the key signal into the key signal storing memory 44. Thus, it can not be said to be excellent in cost merit to use a high speed CPU as such a general-purpose CPU. That is, although a general-purpose CPU becomes expensive as its speed becomes high, only part of the increase of cost is reflected to the increase of speed of generation of the key signal.
In the conventional video switcher, as shown in FIG. 3, for example, key signal generating programs for generating key signals with predetermined shape patterns are stored in the program memory 43 as an internal ROM or the like, and a key signal generating program corresponding to a key number supplied from the controller 31 is executed, so that a key signal with a predetermined shape is generated. Thus, it is difficult to generate a key signal with a shape corresponding to a key signal generating program which is not stored in the program memory 43, and it is not easy to add a key signal generating program for generating a key signal with a new shape into the program memory 43 of the ROM. Further, the key signal generating program often depends on the inner structure of the key signal generating unit 32 as a platform, that is, especially on the CPU 42, and its portability is low. Thus, for example, it is poor in productivity to prepare a key signal generating program for generating a key signal which has a shape and the like meeting the request of the user, and a result, the key signal generating program becomes expensive. Further, in the case where a new key signal generating program is added, a key selecting program (FIG. 3) for giving a key number and parameter to the key signal program must be also changed, so that management of programs becomes troublesome.
From the above, in the conventional video switcher, it has been actually impossible to generate a key signal with a shape and so forth corresponding to a key signal generating program which is not stored in the program memory 43.
Further, as shown in FIG. 8, in the case where a key signal K of such an animation that a star rises in the direction from the depth side to this side is generated, according to the foregoing, it is necessary that a key signal generating program for forming the animation is stored in the program memory 43. However, since the program memory 43 stores programs for generating key signals of other shapes, the size of data of one program is restricted. On the other hand, a program for rendering such a three-dimensional image that the star rises in the direction from the depth side to this side has generally a large amount of data, and it is difficult to store such a program in the program memory 43. Thus, in the conventional video switcher, such an animation that the star rises in the direction from the depth side to this side is frequently substituted by a two-dimensional animation such that a collapsed star in the up-and-down direction gradually extends in the up-and-down direction as shown in FIG. 8.
Besides, as a method of realizing a three-dimensional animation, there is such a method that calculation for generating a three-dimensional image is carried out on the basis of patterns of some predetermined motions, a table containing its result converted into two-dimensional system one is prepared, and a two-dimensional key signal is generated using the table. However, in this method, it is impossible to realize motions other than those registered in the table, so that a mixed image obtained by this is poor in the degree of freedom of expression.
The present invention has been made in view of such circumstances, and an object of the invention is therefore to provide an image processing apparatus in which various key signals can be generated at high speed and further, by this, a mixed image with high degree of freedom of expression can be obtained.
According to a first aspect of the present invention, an image processing apparatus for mixing a first image and a second image comprises: rendering command generating means for generating a rendering command for setting a three-dimensional shape defined by at least one unit figure; an image memory; rendering means for rendering a three-dimensional image corresponding to the three-dimensional shape onto the image memory based on the rendering command; and output means for outputting the three-dimensional image rendered on the image memory as a key signal; mixing means for mixing said first image and said second image in accordance with the key signal.
In the image processing apparatus of the first aspect, the rendering means is a three-dimensional rendering unit.
Beside, in the image processing apparatus, the three-dimensional image is formed of a plurality of polygons.
Besides, in the image processing apparatus, the rendering command includes at least coordinate and color information of each vertex of the polygons in a three-dimensional space.
According to a second aspect of the present invention, an image processing apparatus for generating a key signal as a signal for controlling transmittance of an image when a first image and a second image are mixed, comprises rendering command generating means for generating a rendering command for setting a three-dimensional shape defined by at least one unit figure; an image memory; rendering means for rendering a three-dimensional image corresponding to the three-dimensional shape onto the image memory based on the rendering command; and output means for outputting the three-dimensional image rendered on the image memory as the key signal.
In the image processing apparatus of the second aspect, the rendering means is a three-dimensional rendering unit.
Besides, in the image processing apparatus, the three-dimensional image is formed of a plurality of polygons.
Besides, in the image processing apparatus, the rendering command includes at least coordinate and color information of each vertex of the polygons in a three-dimensional space.
According to a third aspect of the present invention, an image processing apparatus for mixing a first image and a second image comprises: an image memory; receiving means for receiving a rendering command for setting a three-dimensional shape defined by at least one unit figure; rendering means for rendering a three-dimensional image corresponding to the three-dimensional shape onto the image memory based on the rendering command; and output means for outputting the three-dimensional image rendered on the image memory as a key signal; mixing means for mixing said first image and said second image in accordance with the key signal.
In the image processing apparatus of the third aspect, the rendering means is a three-dimensional rendering unit.
Besides, in the image processing apparatus, the three-dimensional image is formed of a plurality of polygons.
Besides, in the image processing apparatus, the rendering command includes coordinate and color information of each vertex of the polygons in a three-dimensional space.
According to a fourth aspect of the present invention, an image processing method for mixing a first image and a second image comprises the steps of: generating a rendering command for setting a three-dimensional shape defined by at least one unit figure; rendering a three-dimensional image corresponding to the three-dimensional shape onto the image memory based on the rendering command; outputting the three-dimensional image rendered on the image memory as a key signal; and mixing the first image and the second image in accordance with the key signal.